Approximation Strategies for Routing in Stochastic Dynamic Networks

In this work, we study a special semi-Markov decision process that formalizes a route-planning problem in stochastic transportation networks. We explore two versions of the planning problem: one in which the planner knows the initial traffic situation but does not have access to further information once it begins executing the plan (open-loop), and one in which the planner receives continuous traffic updates (closed-loop). Since exact versions of these planning problems are intractable, we study their heuristic approximations. We present a new class of Monte-Carlo route planning algorithms that optimize the route using a sample of multiple traffic-state trajectories. We show that these methods are able to outperform Monte-Carlo methods based on greedy policy look-aheads applied most frequently to solve the stochastic decision problems.